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Abstract 

This study explored an alternative assessment procedure to examine learning trajectories of matrix multiplication. 
It took rule-based analytical and cognitive task analysis methods specifically to break down operation rules for a 
given matrix multiplication. Based on the analysis results, a hierarchical Bayesian network, an assessment model, 
comprising of 2 layers of explanatory variables-Matrix Multiplication, Performance and Semantic Explanations; 
and one layer of evidential variables containing 9 evidential variables-was developed. With the simulating data, 9 
students’ Performance and Semantic Explanation evidences were recorded. The results indicated that the 
hierarchical Bayesian assessment effectively traced and recorded students’ learning trajectories; and assessed 
students’ learning dynamically and diagnostically. 

Keywords: Bayesian network, cognitive assessment, cognitive task, matrix multiplication, rule-based analytical 
model, structural representation, vector 

1. Context and Research Problem 

The Mastery of matrix operations was a necessary step for graduate students in education and other social 
sciences to understand the data analytical techniques in their quantitative research designs and data analyses 
(Poole, 2011; Stevens, 2009). Some graduate students and doctoral candidates registered in their programs with 
different educational backgrounds coupled with insufficient preparation in research design and data analysis. 
Moreover, they lack adequate knowledge of advanced linear algebra, such as matrix operations, which are 
fundamentals in quantitative research method learning. 

From learning science point of view, a cognitive process model is required to both improve learning proficiency 
and provide effective assessment information. Cognitively diagnostic assessment model is such a model that can 
be used to attain the above goals (Almond, Mislevy, Steinberg, Yan, & Williamson, 2015; Rupp, Templin, & 
Henson, 2010; Lu & Zhang, 2013; Mislevey, 1995; Zhang & Frederiksen, 2007; Zhang & Leung, 2007; Zhang & 
Lu, 2014a, 2014b). The diagnostic information is the feedback to inform learners where and what steps they 
have to improve, what rules they have to apply to the problem solving and what effective learning trajectory they 
had better follow (Lajoie, 2003; Zhang & Lu, 2012). Thus, a diagnostic assessment model can integrate learning 
strategies, learning trajectories and effectively diagnostic information into one. The Bayesian network is an 
appropriate method hierarchically used to represent the structure of the cognitively diagnostic assessment model. 
Thus, cognitively diagnostic assessment model with Bayesian network representation is an appropriate 
description of this study. 

2. The Study Purposes 

This study is to explore an effect assessment procedure to describe cognitive trajectories and diagnose the 
learning problems, specifically the problems graduate students have in their matrix operation problem solving 
opportunities. Thus, how successfully graduates students master matrix operations and how skillfully they apply 
the matrix knowledge in the data analysis process are crucial concerns. Further, it is important to develop an 
effective assessment procedure to recognize the problems in the matrix operation processes. 

3. The Content Focus on the Algebra Learning Tasks 

In this study, we take matrix multiplications as the learning task. In Matrix multiplication, there are three 
elements involved the multiplication operations: Scalar, Vector and Matrix. A real number or a constant is 
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referred to as a scalar such as 8, 2.58, x/3 and n etc. A vector, algebraically, is a set of ordered n-tuples of real 
numbers written in either a row vector such as v’=[3, 7, 12, 15, 21] or column vector such as: 


u= 


2 

3 

6 

10 


A matrix is m-by-n dimensional array of real numbers. The individual items in a matrix are called its 


elements or entries. The matrix multiplications are the multiplication operations. The multiplication relations 
include the operations between a scalar and a vector, a scalar and a matrix, two vectors, a vector and a matrix, 
and two matrices. The individual items in a matrix are called its elements or entries such as: 


3 2 5 1 
14 2 6 
2 9 7 3 
8 2 17 
2 3 4 3 


. The matrix can generally be written as 


dll d 12 

Clix Cl 22 


Cl 1 n 

Ct 2 n 


Cl ml Cl 


m2 


a r 


For each 


a* 


, the subscript i indicates row and j indicates column. For example, 


a 


21 


is the element in 


second row and first column. 


4. The Rules of Matrix Multiplications 

There are a set of rules in matrix multiplications used to guide the learners to solve the problems. A vector is 
designated by black fond lower case u, v or other lowercase letters; a matrix is designated by an uppercase letter 
A, B or other uppercase letters. 

1) cv=c[v 1, v 2 . . V n ]= [cv 1, cv 2 . . cv n ] 


2) cA= 


ca n ca 
ca 2i ca 


12 


22 


Cdln 

ca2n 


ca m i ca 


m2 


ca r 


3) In the vector multiplication, the column dimension of the first vector should be equal to the row dimension of 
the second vector. In other words, the multiplication between two vectors should consist of one row vector v’ 
and one column vector u. 

4) The product of two vectors v’ and u is defined as ^ = XvVw i' 

5) The product of two vectors u and v’ is defined as ^ = y/ f 


6) v’u^uv’. v’u is a scalar, uv’ is a matrix which can be designated by A ik . The row dimension of the matrix A is 
determined by the row dimension of the u; the column dimension of the matrix A is determined by the column 
dimension of the v\ In order to further understand this rule, I use two examples to show the results of v’u and 
uv’. 
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Assuming there are two vectors, v’= [2, 4, 5] and u= 


3 
1 

4 


, the product A of v’u is that A=[2x3+ 4x1+ 5x4]=30. 


For the same two vectors we can exchange the position of the two vectors as uv% the product is B: 


B =uv' = 

"3" 

1 

[2, 4, 5] 

“3x2 3x4 3x5“ 

1x2 1x4 1x5 

- 

“6 12 15" 

2 4 5 


4 


4x2 4x4 4x5 


8 16 20 


7) There are vector w, w’ and matrix Q. The multiplication between one vector and a matrix are w’Q and Qw. 


w’Q is not equal to Qw. We define ^ is the product of S = ' A - Sjr'EQj, 


Wn- 


The products can be illustrated with examples. 



"l“ 


“l 

2 f 

Assuming there are vectors w’=[l, 2, 3], w = 

2 

and matrix Q = 

2 

4 3 


3 

3 

5 1 


1 2 1 

S li = Zw',,0 # r"’'= 2. 3] 2 4 3 

[3 5 1 

= [lxl + 2x2+3x3, Ix2 + 2x4+3x5, lxl + 2x3 + 3xl] 
= [14, 25, 10] 

S j\ -lLQ Jk Wk\ 


8) The multiplication between two matrices requires that the column dimension of the first matrix be equal to the 
row dimension of the second matrix. 

There are matrices X and Y. the product of these two matrices Z is defined as 

Z ik = X ij Y jk 


1 

2 

1“ 


V 


Ixl+2x2+lx3 


“ 8 “ 

2 

4 

3 


2 

= 

2xl+4x2+3x3 

= 

19 

3 

5 

1 


3 


3xl+5x2+lx3 


16 


Assuming that 


"l 1 2“ 

2 3 1 

, and Y = 

“2 3“ 

4 1 

- 


1 2 


, the product of these two matrices: 





“2 

3“ 








“1 1 2“ 





Ix2 + lx4 + 2xl 

Ix3+lxl+2x2 


“8 

8“ 

Z = 

2 3 1 


4 

1 

1 

- 

2x2+3x4+lxl 

2x3+3xl+lx2 

- 

17 

11 




2 








9) The positions of vectors/matrices in composite matrix multiplications cannot be changed. 
For example: v ? V u is a scalar. 

v 1 m mn vt n \ 


5. Relevant Knowledge Supporting Matrix Multiplication 

Matrix multiplication learning is at the advanced level of research study design and data analysis. It is assumed 
that the learners have all mathematics, geometries and statistics knowledge to support the learning task. All 
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pieces of knowledge are potential problem solving rules and conditions. There is weak association between the 
relevant knowledge and completion of matrix multiplication. 

1) The learners are able to understand, R n which is the set of all ordered n-tuples of real numbers written as row 
or column vectors. Thus a vector v in R n is of the form 


v, 

v 2 


[vi> V 2 ’”vJ or 


V 


n. 


2) The learners understand the operations of addition, subtraction and multiplication of scalars. 

3) The learners understand positive numbers and negative numbers. 

4) The learners are able to understand and do calculations of sum of products. 

All these above consist of the supplements of the rules of matrix multiplications. At the advanced learning level, 
amount of the knowledge and problem solving skills to be prepared to solve matrix multiplication is unknown. 
The mistakes and non-proficiency can receive the evidence from a specific study based on a specific learner 
group. 

6. Rule-Space Model and Rule-Based Analytical Model 

6.1 Rule-Space Model 

Tatsuoka (1983) developed the rule-space model, which was the combination of cognitive task analysis and 
psychometric measurement models (Snow & Lohman, 1989). The rule space was explored by using cognitive 
task analysis (Clark & Estes, 1996) to obtain a set of attributes which represent a set of specific skills or 
competencies (Gierl, Leighton, & Hunka, 2000). A typical example is fraction operation where Q matrix was 
applied to (Tatsuoka, 2009). The Q matrix provides a two-dimension structure to highlight a set of rules being 
written as elements in the matrix to represent a given underlying cognitive structure. The elements consist of the 
skills/attribute set. Each cognitive task (/test item) requires some or all of elements within the Q matrix. 
Mathematically the J x K g-matrix can be defined with the following descriptions. 

Q jk = 1 or 0 depending on whether or not attribute k is required by task j. 

In the algebra multiplication learning task, it assumes that there are i students. 

Xy = 1 or 0 depending on whether or not student i performs task j correctly 
a ik = 1 or 0 depending on whether or not student i possesses attribute k. 

Thus, the g-matrix forms the basis of the cognitive diagnostic assessment structure. 

Here, an arbitrary Q-matrix is shown for the illustrated purpose. 

In this two-dimension table there are 5 learners and they face 5 attribute elements to solve fraction problem, for 

example. We can see that learner 1 has attribute 3 and 4, learner 2 has attribute 1, 2 and 5; learner 3 only has 

attribute 1 and 5; learner 4 has attribute 3, 4 and 5. Lastly learner 5 has all of 5 attributes. 


Q 


5x5 



Attr 1 

Attr 2 

Attr 3 

Attr 4 

Attr 5 

learner 1 

0 

0 

1 

1 

0 

learner 2 

1 

1 

0 

0 

1 

learner 3 

1 

0 

0 

0 

1 

learner A 

0 

1 

1 

1 

0 

learner 5 

1 

1 

1 

1 

1 
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Q-matrix has two properties: a) the proficiency status is dichotomous, 0 and 1; and the rule space is closed. 
Stated differently, there are only two statuses for each learner to complete the fraction cognitive task: success and 
fail; the cognitive task is well-structured and all of the rules are well-defined. Thus, the rule space is closed. Such 
ideal problem structure is possible in the elementary level of algebra operation and learning. However, at 
advanced level of algebra, operation such as matrix multiplication is not appropriate. Multi-statuses for the 
cognitive task completion are required for both learning strategies and diagnostic assessment. Further, the rule 
space is not closed. Beyond the rules of matrix multiplication, there are some potential knowledge and problem 
solving skills regarded as cognitive attributes, candidates must possess. Thus, an alternative model—rule-based 
analytical model is necessary to be developed. 

6.2 Rule-Based Analytical Model 

Rule-based analytical model is a mixed model from rule space model and cognitive task analysis (Clark, Feldon, 
van Merrienboer, Yates, & Early, 2007; Graco, 2012). How to establish a cognitive model in a diagnostic 
assessment is dominated by the characteristics of the cognitive task itself. It is cognitive task specific. When we 
solve fundamental algebra problems such as fraction problem, a closed rule space model with dichotomous 
attributes is sufficient. When a cognitive task is completely ill-structured such as learning in medical emergency 
environment, cognitive task analysis approach is strongly suggested (Zhang & Lu, 2014b). The operation in 
matrix multiplication requires both a set of rules and cognitive task analysis. In other words, the problem space 
of attributes is opened rather than closed because the rules cannot be exhausted and relevant problem solving 
knowledge and skills may become attribute elements with the analysis of learners’ evidence of matrix 
multiplication, especially those mistake evidence. 

7. Research Questions 

The critical issue in this study is how to establish an effect diagnostic assessment procedure for learners in 
matrix multiplications. This effective assessment model can provide both sufficient learning strategies, and 
diagnostic assessment information when they are challenged by some problem ran across in the operation of 
matrix multiplications. Thus, the following research question can be addressed: 

1) How the rules of matrix multiplication can be used to develop rule based analytical model in the diagnostic 
assessment? 

2) How the cogitative task analysis can be used in setting up a cognitive diagnostic model? 

3) How are the learners’ knowledge and problem solving skills in matrix multiplication assessed sufficiently 
and reported diagnostically? 

8. Methodology 

The research methodology includes both theoretical framework, data-driven model analytical process. The 
theoretical framework describes the rationale how to develop an assessment model which reflects both learning 
and assessment process. The data-driven model analytical process depict the step-by-step process in the 
assessment model development 

8.1 Rule-Based Model and Cognitive Task Analysis 

There are 8 rules for the matrix multiplication. As the guide to develop evidential variable, the set of learner’s 
rules can be referred to building the diagnostic assessment framework. Briefly, these rules and relevant 
knowledge are summarized here again: 

1) cv: the multiplication of scalar c and vector v. 

2) cA: the multiplication of scalar c and matrix A. 

3) v’u or uv’: the production of two vectors. 

4) v’u^uv’. v’u is a scalar, uv’ is a matrix 

5) For w, w’, two vector and matrix Q, w’Q^Qw. 

6) For the product ^ ^ Y ^ , the dimensions should be effectively defined. 

In addition, some relevant knowledge should be proficient for the learner, such as concept and problem solving 
skills of the sum of products: ^ .. 

7) The positions of vectors/matrices in composite matrix multiplications cannot be changed. 
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8.2 Diagnostic Assessment Model and Structured Representation 

The matrix multiplication is an advanced operation in mathematics computation. Even there are several rules 
being recognized, the all sample space of the problem solving cannot be exhausted. It is better to take an 
example to illustrate the development of the diagnostic assessment model and further represent the assessment 
structure. 

Supposed there are two matrices Ra imd S i2 . The product is T mn an d the learner is asked to get the 
solution to the matrix multiplication. 


Generally we have J ^ 


33 


Y 11 Y 12 V 13 

Yn r 2 i Y23 

r 3 1 r 3 2 r 33 . 


and C = 

O 32 


S 11 S 12 
S 21 S 22 

S 31 S 32. 


, Now we have two specific matrices: 


R 33 - 


1 1 2 

2 3 1 
1 2 1 


S» = 


T m n ~ R. 33 X S 32 = 


32 

1 1 2 ' 

2 3 1 

1 2 1 


1 3 
4 2 

2 3 

1 3' 
4 2 

2 3 


and 


now 


the learning task 


is 


to 


gam 


The rule and knowledge to complete this matrix multiplication at the moment include: 

1) To be able to determine the definition, applying to Rule 4 and 8 of Section 4. 

2) To be able to know that the sum of product of y ^ and g ^ is the element ^ ^ of new matrix. 

3) To be able to know T 3 4 5 6 7 =7^ 

7 1 mn 1 32 

4) To be able to perform the product of each pair y and g ^ , such a - s J^ n x S n • 


5) To be able to perform the sum of the product ^ ^ for each y and g ^ such as: 


f = 7 T/* C» - T/* Xo +T^ Xo + jr Xo 

i 11 Lull lyOyl — r 11 011 A 12 0 21 A 13 0 31 


6) To be able to explain the subscripts for both y and g 


Based on these knowledge and cognitive trajectory analysis, we attempt to establish a hierarchical cognitive 
process model. The model consists of two cognitive aspects: performance and semantics explanations. Under 
each aspect following the cognitive trajectory, we can establish the model which is hierarchical. The model is 
conceptually represented via BayesiaLab 6.02 (Conrady & Jouffe, 2015) because Bayesian network model can 
be multi- representations. At this level, it is only a hierarchally conceptual representation of the diagnostic 
assessment model of Matrix Multiplication. 
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Figure 1. Conceptually hierarchical model of matrix multiplication 


There are three levels of this hierarchical diagnostic assessment model (see Figure 1). The first level is the 
proficiency of Matrix Multiplication. The second level consists of two elements: Performance and Semantic 
Explanation. Further, the Performance has three subcomponents as evidential variables; and the Semantic 
Explanation consists of six subcomponents as evidential variables. In total, there are nine evidential variables. 
Matrix Multiplication, Performance and Semantic Explanation are called explanatory variables (Zhang & Lu, 
2014). At the model, this is an “empty” model which does not contain any quantitative information. In next 
section, the quantitative information will be added. 

8.3 Bayesian Network as Representation Structure 

Bayesian network is a good tool to represent a cognitive or learning process, based on concept of a declarative 
representation. The key property of a declarative representation is the separation of knowledge and reasoning. It 
can be both probabilistic graphical model and structured probabilistic model. Koller and Friedman (2009) 
present two different models clearly and logically: conceptual representation and quantitative representation. The 
two representations can be meantime visualized via one probabilistic graphical model. Figure 2 only provides a 
conceptual structure of matrix multiplication diagnostic assessment model. In order to further quantify the model, 
we have to initialize the proficiency level for each variable (Zhang & Lu, 2014). 

8.3.1 Initializing Values of the Bayesian Network Diagnostic Assessment Model 

Cognitively, two explanatory variables are defined: Performance and Semantic Explanation. In both performance 
and semantic explanation models, all variables are technically called nodes (Koller & Friedman, 2009). Learners’ 
knowledge and skills in solving matrix multiplication problem are expressed as both explanatory and evidential 
variables. The explanatory variables receive either performance or explanatory information via the input of the 
evidential variables. However, all nodes should be initialized before accepting learners’ information of 
performance and semantic explanations. 

There are several different ways to initialize the Bayesian network model. Statistical facts, expert beliefs and 
experiences are important resources. In this study, we do not have literature to report learner’s performance and 
semantic explanation records in matrix multiplication; though, we can find the literatures of diagnostic 
assessment and Bayesian network (Zhang, 2007). Thus, we arbitrarily take successful probability for Matrix 
Multiplication as 0.67 and fail probability as 0.33 which are believes rather than ones from evidences (see Figure 
2(a)). 


Matrix Multiplication v 


sses Values State Names Reference State 


Fail Success 

33.0001 67.000 


Figure 2(a). The initial belief of matrix multiplication 
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When we further look at the conditional probabilities of Performance, the chance to receive probability fail under 
the condition of Matrix Multiplication fail is 0.67. The chance to receive probability success under the condition 
of Matrix Multiplication success is also 0.67. The chance to receive probability fail under the condition of Matrix 
Multiplication success is 0.33. The chance to receive probability success under the condition of the Matrix 
Multiplication fail is also 0.33 (see Figure 2(b)). 


n: Performance >✓ 


lasses Values State Names Reference State F 


Matrix Mult... Fail 

Fail 67.000 

Success 

33.000 

Success 33.000 

67.000 



Figure 2(b). Conditional probability of matrix multiplication 


The different initialized values only present a difference between different models at very beginning level of the 
model. With the sample increasing, the probability level will be convergent to the theoretical average level. In 
the following matrix multiplication diagnostic assessment model, there are Performance and Semantic 
Explanation, where the initiative probability values are called conditional probabilities that are conditional on the 
Matrix Multiplication (Koski & Noble, 2009; Korb & Nicholson, 2011). That means the information is 
conditional on the upper nodes. When all variables/ nodes consisting of the model has not yet received any 
evidence information, the model is called initialized template model (see Figure 3). 



Figure 3. Initialized template model of matrix multiplication assessment 


8.3.2 Joint Probability of the Bayesian Network Assessment Model 

Joint probability is the probability of two or more events occurring together. If there are event A and event B, the 
probability of the intersection of event A and event B may be written p (A D B). For example we can focus on 
the joint probability of the event that matrix multiplication success and performance success together. Figure 4 
presents the joint probabilities of three variables/ nodes, Matrix Multiplication, Performance and Semantic 
Explanation. There is no any condition of Matrix Multiplication, so the joint probability is just the same as the 
initial values. Under the condition of Matrix Multiplication, the joint probabilities of Performance are 0.5578 for 
success and 0.4422 for fail. There is not big difference between success and fail because there is no any evidence 
to update the network model. Another variable, Semantic Explanation, indicates the same probability distribution 
for both success and fail. All other evidential variables are also initialized via the same way. 
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Matrix Multiplication 


33.00% [ 

Fail 

67.00% [ — 

Success 





Performance 



44.22% 1 

Fail 


55.78% [ 

Success 






Semantic Explanation 


44.22% 

55.78% 



Fail 

Success 


Figure 4. Joint probabilities of matrix multiplication, performance and semantic explanation 


8.3.3 Evidences Received via Evidential Variable Updating 

In a diagnostic assessment model with Bayesian network, there are two types of variables: explanatory variable 
and evidential variable. In Matrix Multiplication model of this study, there are three explanation variables: 
Matrix Multiplication, Performance and Semantic Explanation. There are nine evidence variables. Performance 
on the Product rij&sjk, Performance on the Sum of rij&sjk production, To Find Each tjk’s Place, Determining 
mxn of T, Explaining the Definition of T, Explaining Subscripts of All Terms, Explaining rij, Explaining sjk and 
Explaining rij.sjk (see Figure 5). 



8.3.4 Updating Probabilities of Explanatory Variable with Random Evidence 

There are nine evidential variables which can be used to update Matrix Multiplication diagnostic assessment 
model. A randomly sampling method is applied to test evidence states from zero success to nine success 
evidence observations. The success status means that a learner receives a positive score of the given 
variable—either performance or semantic explanation. 

Table 1 presents the relations of instantiated evidential variable and explanatory variables. Matrix Multiplication 
indicates the general level of a matrix multiplication problem. The minimum value for Matrix Multiplication is 
0.37, Performance is 0.1074 and Semantic Explanation is 0.0148. The maximum value for Matrix Multiplication 
is 0.8747, Performance is 0.9296 and Semantic Explanation is 0.9904. The minimum and maximum values 
indicate the range of each variable. For example, the range of Matrix Multiplication is 0.8747-0.3713=0.5034. 
Further, we observe the increase of the values of each variable with the evidences increase. We expect a robust 
trend of mono increase for the learning curve. We observe that evidence numbers equal to and greater than 7 
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while the updated values of each column keep a mono increase. Thus, it seems it is plausible we say when the 
learner masters 7 and 7 + score points, the learner’s progress becomes robust. Of course, such a conclusion is 
model specific. Different cognitive and assessment models will have different cognitive trajectory patterns. 

Table 1. Updated probabilities of random evidence variables 


Number of Positive Evidence 

Matrix Multiplication 

Performance 

Semantic Explanation 

0 

0.3713 

0.1074 

0.0148 

1 

0.4475 

0.3316 

0.0165 

2 

0.4631 

0.3364 

0.0646 

3 

0.5138 

0.3518 

0.2215 

4 

0.7102 

0.4117 

0.8286 

5 

0.7784 

0.9135 

0.5947 

6 

0.7192 

0.7192 

0.5745 

7 

0.8270 

0.7520 

0.9582 

8 

0.8676 

0.9259 

0.9614 

9 

0.8747 

0.9269 

0.9904 


9. Students’ Proficiency in Matrix Multiplication Problem Solving 

There is a simulated data set assuming it consists of 9 individual learners. Sometimes, there is an unbalance in 
the learning tasks of performance and semantic explanation. There are at least two reasons for such a phenomena: 
a) the learners’ individual experience bias and b) the property of the learning tasks. Some learning task is easier 
to perform and challenging to explain semantically, and some learning task is easier to explain, but it is very 
difficult to perform. Table 2 records a simulation records that reflect 9 students’ performance. The patterns of 
learners’ scores are very interesting. Subjects 1, 2 and 3 show that their Performance scores very well, but not do 
well in Semantic Explanation. Subjects 4, 5 and 6’s scores on Matrix Multiplication indicate that they score on 
Semantic Explanation very well, but do not do well in Performance. Subjects 7, 8 and 9 present balance patterns. 
Subject 7 receives balance achievement among both Performance and Semantic Explanation at lower level; 
Subject 8 receives balance score in both Performance and Semantic Explanation with intermedia scores; and 
subject 9 receives highest score in both Performance and Semantic Explanation. 


Table 2. Students’ proficiency of matrix multiplication 


Subject Number 

Matrix Multiplication 

Performance 

Semantic Explanation 

1 

0.6976 

0.9022 

0.2625 

2 

0.7192 

0.7192 

0.5745 

3 

0.8425 

0.92.24 

0.8581 

4 

0.6412 

0.1451 

0.8132 

5 

0.4239 

0.4239 

0.9522 

6 

0.7618 

0.4274 

0.9880 

7 

0.7959 

0.7426 

0.8477 

8 

0.8270 

0.7520 

0.9582 

9 

0.8747 

0.9269 

0.9904 


10. Conclusions and Discussion 

This study examined the processes and procedures of developing a diagnostic assessment model with Bayesian 
network representation for matrix multiplication learning. The diagnostic assessment model received a 
conceptual and quantitative representation with the Bayesian network model. The study presented a step-by-step 
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process to display how to develop an effective diagnostic assessment model of matrix multiplication by using 
rule-based analytical model and the techniques of cognitive task analysis. The rules of matrix multiplication and 
relevant results of cognitive task analysis were integrated into the assessment element representation in the 
diagnostic assessment model with Bayesian network. The model had three exploratory variables which represent 
the learners’ proficiency in solving matrix multiplication problems. The proficiency was represented in two 
aspects: performance and semantic explanations. The learners should be able to know matrix multiplication rules 
and apply these rules in their own practice in solving matrix multiplication problems. Further the learners were 
able to explain the rationale of matrix multiplication rules and the relations of these rules. There were 9 
evidential variables by which learner’s score evidence could be updated through the evidential variables to upper 
variables in the Bayesian network assessment model—Performance and Semantic Explanation, and then to 
Matrix Multiplication. Statistically, this hierarchical diagnostic assessment model was non-linear and provided 
an effective assessment tool to measure and assess learners’ performance and semantic explanation of Matrix 
Multiplication. 

The diagnostic assessment model successfully assessed learners’ progress, and it also effectively differentiated 
cognitive aspects into performance and semantic explanations. The non-linear hierarchical assessment model 
could report the mastery proficiency at macro cognitive level in Matrix Multiplication; and further the mastery 
proficiency at sub-cognitive levels in both Performance and Semantic Explanation. Sometimes both 
sub-cognitive proficiencies presented unbalance status even though the macro cognitive level did not indicate 
much difference in two or three different cases. In study, Performance indicated a very high level, and Semantic 
Explanation scored very low, or vice versa. 

The hierarchical diagnostic assessment model for Matrix Multiplication also effectively differentiates 
proficiencies among different learners, and meantime, the model provides dynamic diagnostic information. It 
allows the learners to understand what and where they perform unskillfully. 

Lastly, the hierarchical diagnostic assessment model for Matrix Multiplication indicates a robust learning 
progress trajectory which can be described as learning progress curve. From Table 1, we conclude that the 
learners should master at least 7 knowledge elements based on the rule-based analysis, thus, the learning 
becomes more robust. The development process of the hierarchical diagnostic assessment model for Matrix 
Multiplication can be transferred to other academic learning assessment domains. 

11. Limitations 

The hierarchical diagnostic assessment model for Matrix Multiplication was simulated with 9 learners’ data. The 
conclusions were limitedly generalized to different sample groups. The model was also cognitive task structure 
specific. The exact models for different cognitive tasks are not expected. However, the matrix multiplication, as 
an example, presents an effective model for statistical and research method researchers to continue to explore 
valid learning and assessment model. 
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